Embodiments of the present invention relate to a new approach to support video communication with the provision of eye contact. Some embodiments relate to a virtual camera pose determiner. Some embodiments relate to a video communication system. Some embodiments relate to a method for determining a position and an orientation of a virtual camera. Some embodiments relate to a method for video communication.
A significant drawback of current video communication systems is the lack of eye contact. This is caused due to fact that the user is looking onto the screen to the remote participant (to the displayed image of the remote participant), while the person is captured by a camera mounted on top of the display. A well-known approach is to provide a novel view of the remote participants based on a virtual camera, which is placed at the position where the users are looking at on the display. However, such a virtual camera is always looking straight in a fixed direction, whether the local participant is in the field of view of the camera or not. This may result in a “static” or unnatural appearance of the local participant when being rendered at the remote site.
U.S. Pat. No. 7,532,230 B1 discloses a method of communicating gaze in an immersive virtual environment. The method begins by representing an immersive virtual environment on a monitor that is viewable by a viewing participant. The monitor displays a plurality of objects in the virtual environment. The physical gaze of the viewing participant is tracked to determine a physical direction of the physical gaze within a physical environment including the viewing participant. Thereafter, a viewed object is determined at which the viewing participant is gazing. Then, a virtual direction is determined between the viewing participant and the viewed object in the immersive virtual environment. A model of the viewing participant is rotated based on the physical and virtual directions to render a view of the viewing participant such that the viewing participant is facing the viewed object in the immersive virtual environment. However, in U.S. Pat. No. 7,532,230 B1, the creation of the correct virtual view is not based on the concept of a virtual camera. Moreover, the concept disclosed in U.S. Pat. No. 7,532,230 B1 requires a full 3D model of the scene. No geometrical constraints are described in U.S. Pat. No. 7,532,230 that are required for eye contact and are based on the line of sight of the local participant and the eye positions of the remote participant shown on the display. U.S. Pat. No. 7,532,230 B1 does not mention that the position, where the user is looking at, may be a mid-eye, a left eye position or a right eye position.